Should Penalized Least Squares Regression be Interpreted as Maximum A Posteriori Estimation?

Nonparametric regression estimation using penalized least squares

We present multivariate penalized least squares regression estimates. We use Vapnik{ Chervonenkis theory and bounds on the covering numbers to analyze convergence of the estimates. We show strong consistency of the truncated versions of the estimates without any conditions on the underlying distribution.

The extended least-squares and the joint maximum-a-posteriori maximum-likelihood estimation criteria

Approximate model equations often relate given measurements to unknown parameters whose estimate is sought. The Least-Squares (LS) estimation criterion assumes the measured data to be exact, and seeks parameters which minimize the model errors. Existing extensions of LS, such as the Total LS (TLS) and Constrained TLS (CTLS) take the opposite approach, namely assume the model equations to be exa...

Classification using partial least squares with penalized logistic regression

MOTIVATION One important aspect of data-mining of microarray data is to discover the molecular variation among cancers. In microarray studies, the number n of samples is relatively small compared to the number p of genes per sample (usually in thousands). It is known that standard statistical methods in classification are efficient (i.e. in the present case, yield successful classifiers) partic...

Penalized Least Squares Regression: Fast Computation via EÆ ient Approximation

Penalized least squares regression methods and applications to neuroimaging

The goals of this paper are to review the most popular methods of predictor selection in regression models, to explain why some fail when the number P of explanatory variables exceeds the number N of participants, and to discuss alternative statistical methods that can be employed in this case. We focus on penalized least squares methods in regression models, and discuss in detail two such meth...